{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "279fbea1-e25b-468c-b0ec-de7a3f3a1615",
   "metadata": {},
   "source": [
    "# 一、赛事背景\n",
    "\n",
    "当代医学诊断实践中，超声成像技术凭借其非侵入性、无放射性、成本效益高及实时性等显著特点，已广泛成为各类医疗场景中不可或缺的诊断工具。该技术通过发射高频声波并捕捉其回波信号，构建出人体内部结构的精确图像/视频，从而为临床医生提供了关于器官、组织和血管系统的详尽信息。\n",
    "\n",
    "尽管如此，超声成像的数据解析与分析过程往往高度依赖于专业医生的主观判断和个人经验，这种依赖可能引发诊断结果的不一致性及误差。此外，众多不同的疾病在超声数据上可能展现出相似的特性，这增加了仅凭视觉检查准确区分和识别特定病种的难度，特别是在多病种共存的情况下。 \n",
    "\n",
    "因此，研究基于超声数据的多病种疾病预测方法显得尤为重要。本次赛事旨在激励全球AI领域研究者开发能够自动处理和分析超声数据，并准确预测患者可能患有的多种疾病的算法和方法。此类技术不仅能提升诊断的准确性和效率，降低误诊或漏诊的风险，而且对优化医疗资源配置、提高诊疗效率、降低医疗成本具有潜在的重大影响。同时，可进一步提升基于超声数据的多病种疾病预测方法的性能，并推动其在实际临床环境中的应用。\n",
    "\n",
    "# 二、赛事任务\n",
    "\n",
    "为研究利用超声数据进行多病种的疾病预测算法，本次大赛提供了数据集训练样本，参赛者需根据提供的超声样本构建模型，对数据集进行处理。\n",
    "\n",
    "# 三、 评审规则\n",
    "\n",
    "**1.数据说明**\n",
    "\n",
    "本次大赛实行一轮赛制，所用数据集图像格式为npy。\n",
    "\n",
    "| 数据类别 | 文件夹名 | 数据格式 | 解释   |\n",
    "| -------- | -------- | -------- | ------ |\n",
    "| 基本数据 | test     | npy      | 测试集 |\n",
    "| 基本数据 | train    | npy      | 训练集 |\n",
    "\n",
    "**2.评估指标**\n",
    "\n",
    "本模型依据提交的结果文件，采用macro F1-score进行评价。\n",
    "\n",
    "**3.评测及排行**\n",
    "\n",
    "1）比赛均提供下载数据，选手在本地进行算法调试，在比赛页面提交结果。\n",
    "\n",
    "2）每支团队每天最多提交5次。\n",
    "\n",
    "3）排行按照得分从高到低排序，排行榜将选择团队的历史最优成绩进行排名。\n",
    "\n",
    "# 四、作品提交要求\n",
    "\n",
    "1、文件格式：按照csv格式提交。\n",
    "\n",
    "2、文件大小：无要求。\n",
    "\n",
    "3、提交次数限制：每支队伍整个赛期最多30次。\n",
    "\n",
    "4、文件详细说明：\n",
    "\n",
    "1) 以csv格式提交，编码为UTF-8，第一行为表头。\n",
    "2) 提交格式见样例。\n",
    "\n",
    "5、排行榜更新结束后，前三名选手需要提交代码、模型和说明文档。\n",
    "\n",
    "# 五、赛程规则\n",
    "\n",
    "本赛题实行一轮赛制\n",
    "\n",
    "**赛程周期**\n",
    "\n",
    "8月19日-9月23日\n",
    "\n",
    "1、8月19日10：00发布训练集、测试集；\n",
    "\n",
    "2、比赛作品提交截止日期为9月23日17：00，公布名次日期为9月下旬。\n",
    "\n",
    "**现场答辩**\n",
    "\n",
    "1、最终前三名团队将受邀参加科大讯飞AI开发者大赛年度总决赛并于现场进行答辩；\n",
    "\n",
    "2、答辩以（10mins陈述+5mins问答）的形式进行；\n",
    "\n",
    "3、根据作品成绩和答辩成绩综合评分（作品成绩占比70％，现场答辩份数占比30％）。\n",
    "\n",
    "# 六、奖项设置\n",
    "\n",
    "本赛题设立一、二、三等奖共三名，具体详情如下：\n",
    "\n",
    "**【奖项激励】**\n",
    "\n",
    "1、TOP3团队颁发获奖证书\n",
    "\n",
    "2、赛道奖金，第一名5000元、第二名3000元、第三名2000元\n",
    "\n",
    "**【资源激励】**\n",
    "\n",
    "1、讯飞开放平台优质AI能力个人资源包\n",
    "\n",
    "2、讯飞AI全链创业扶持资源\n",
    "\n",
    "3、讯飞绿色实习/就业通道\n",
    "\n",
    "注：\n",
    "\n",
    "1.鼓励选手分享参赛心得、参赛技术攻略、大赛相关技术或产品使用体验等文章至组委会邮箱（AICompetition@iflytek.com），有机会获得大赛周边；\n",
    "\n",
    "2.赛事规则及奖金发放解释权归科大讯飞所有；以上全部奖金均为税前金额，将由主办方代扣代缴个人所得税。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ce8b0d03-c17f-4a76-b705-584e6836e649",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "%pylab is deprecated, use %matplotlib inline and import the required libraries.\n",
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    }
   ],
   "source": [
    "import os, sys, glob, argparse\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from tqdm import tqdm\n",
    "\n",
    "%pylab inline\n",
    "\n",
    "import cv2\n",
    "from PIL import Image\n",
    "from sklearn.model_selection import train_test_split, StratifiedKFold, KFold\n",
    "\n",
    "import torch\n",
    "torch.manual_seed(0)\n",
    "torch.backends.cudnn.deterministic = False\n",
    "torch.backends.cudnn.benchmark = True\n",
    "\n",
    "import torchvision.models as models\n",
    "import torchvision.transforms as transforms\n",
    "import torchvision.datasets as datasets\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "from torch.autograd import Variable\n",
    "from torch.utils.data.dataset import Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7acb5621-6503-4be9-b3b1-d89b37c24d72",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_path = glob.glob('./train/*/*.npy')\n",
    "np.random.shuffle(train_path)\n",
    "\n",
    "labels = ['Anomalies', 'Cyst', 'Inflammation', 'Tumor', 'Vascular']\n",
    "train_label = [labels.index(x.split('/')[-2]) for x in train_path]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5b67caa8-4f83-4ade-906f-e9dca1324f86",
   "metadata": {},
   "outputs": [],
   "source": [
    "DATA_CACHE = {}\n",
    "class XunFeiDataset(Dataset):\n",
    "    def __init__(self, img_path, img_label, transform=None):\n",
    "        self.img_path = img_path\n",
    "        self.img_label = img_label\n",
    "        if transform is not None:\n",
    "            self.transform = transform\n",
    "        else:\n",
    "            self.transform = None\n",
    "    def __getitem__(self, index):\n",
    "        if self.img_path[index] in DATA_CACHE:\n",
    "            img = DATA_CACHE[self.img_path[index]]\n",
    "        else:\n",
    "            img = np.load(self.img_path[index])\n",
    "            DATA_CACHE[self.img_path[index]] = img\n",
    "        if self.transform is not None:\n",
    "            img = self.transform(image = img)['image']\n",
    "        img = img.transpose([2,0,1])\n",
    "        return img, torch.from_numpy(np.array(self.img_label[index]))\n",
    "    def __len__(self):\n",
    "        return len(self.img_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e2d68d5e-8696-4126-83d4-d728fc3614c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "class XunFeiNet(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(XunFeiNet, self).__init__()\n",
    "        model = models.resnet18(True)\n",
    "        model.avgpool = nn.AdaptiveAvgPool2d(1)\n",
    "        model.fc = nn.Linear(512, 5)\n",
    "        self.resnet = model\n",
    "    def forward(self, img):\n",
    "        out = self.resnet(img)\n",
    "        return out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c3c911d6-177a-4654-aaaf-81ac29688340",
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(train_loader, model, criterion, optimizer):\n",
    "    model.train()\n",
    "    train_loss = 0.0\n",
    "    for i, (input, target) in enumerate(train_loader):\n",
    "        input = input.cuda(non_blocking=True)\n",
    "        target = target.cuda(non_blocking=True)\n",
    "\n",
    "        # compute output\n",
    "        output = model(input)\n",
    "        loss = criterion(output, target)\n",
    "\n",
    "        # compute gradient and do SGD step\n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        if i % 100 == 0:\n",
    "            print('Train loss', loss.item())\n",
    "            \n",
    "        train_loss += loss.item()\n",
    "    \n",
    "    return train_loss/len(train_loader)\n",
    "            \n",
    "def validate(val_loader, model, criterion):\n",
    "    model.eval()\n",
    "    \n",
    "    val_acc = 0.0\n",
    "    \n",
    "    with torch.no_grad():\n",
    "        end = time.time()\n",
    "        for i, (input, target) in enumerate(val_loader):\n",
    "            input = input.cuda()\n",
    "            target = target.cuda()\n",
    "\n",
    "            # compute output\n",
    "            output = model(input)\n",
    "            loss = criterion(output, target)\n",
    "            \n",
    "            val_acc += (output.argmax(1) == target).sum().item()\n",
    "            \n",
    "    return val_acc / len(val_loader.dataset)\n",
    "\n",
    "def predict(test_loader, model, criterion):\n",
    "    model.eval()\n",
    "    val_acc = 0.0\n",
    "    \n",
    "    test_pred = []\n",
    "    with torch.no_grad():\n",
    "        end = time.time()\n",
    "        for i, (input, target) in enumerate(test_loader):\n",
    "            input = input.cuda()\n",
    "            target = target.cuda()\n",
    "\n",
    "            # compute output\n",
    "            output = model(input)\n",
    "            test_pred.append(output.data.cpu().numpy())\n",
    "            \n",
    "    return np.vstack(test_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a6521197-3da3-40da-b2e3-2e4d579f6231",
   "metadata": {},
   "outputs": [],
   "source": [
    "import albumentations as A\n",
    "train_loader = torch.utils.data.DataLoader(\n",
    "    XunFeiDataset(train_path[:-100], train_label[:-100],\n",
    "            A.Compose([\n",
    "            A.HorizontalFlip(p=0.5),\n",
    "            A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225))\n",
    "        ])\n",
    "    ), batch_size=30, shuffle=True, num_workers=1, pin_memory=False\n",
    ")\n",
    "\n",
    "val_loader = torch.utils.data.DataLoader(\n",
    "    XunFeiDataset(train_path[-100:], train_label[-100:],\n",
    "            A.Compose([\n",
    "            # A.HorizontalFlip(p=0.5),\n",
    "            # A.RandomContrast(p=0.5),\n",
    "            A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225))\n",
    "        ])\n",
    "    ), batch_size=30, shuffle=False, num_workers=1, pin_memory=False\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "1c758b1c-a00b-4c44-a3a3-5aa4022b900e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/lyz/anaconda3/envs/py311/lib/python3.11/site-packages/torchvision/models/_utils.py:135: UserWarning: Using 'weights' as positional parameter(s) is deprecated since 0.13 and may be removed in the future. Please use keyword parameter(s) instead.\n",
      "  warnings.warn(\n",
      "/home/lyz/anaconda3/envs/py311/lib/python3.11/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.\n",
      "  warnings.warn(msg)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train loss 1.6971927881240845\n",
      "0.7282405907909075 0.655793025871766 0.71\n",
      "Train loss 0.16219213604927063\n",
      "0.19310481796662013 0.9617547806524185 0.95\n",
      "Train loss 0.07772817462682724\n",
      "0.08410332364340624 0.8807649043869517 0.85\n",
      "Train loss 0.166463702917099\n",
      "0.23089910261332988 0.7851518560179978 0.72\n",
      "Train loss 0.09972250461578369\n",
      "0.13735906984657048 0.8177727784026997 0.83\n",
      "Train loss 0.08735839277505875\n",
      "0.06372769172303379 0.9853768278965129 0.99\n",
      "Train loss 0.062059804797172546\n",
      "0.024383619807971022 1.0 0.99\n",
      "Train loss 0.009021206758916378\n",
      "0.013137612421996892 0.9988751406074241 0.99\n"
     ]
    }
   ],
   "source": [
    "model = XunFeiNet()\n",
    "model = model.to('cuda')\n",
    "criterion = nn.CrossEntropyLoss().cuda()\n",
    "optimizer = torch.optim.AdamW(model.parameters(), 0.001)\n",
    "\n",
    "for _  in range(8):\n",
    "    train_loss = train(train_loader, model, criterion, optimizer)\n",
    "    val_acc  = validate(val_loader, model, criterion)\n",
    "    train_acc = validate(train_loader, model, criterion)\n",
    "    print(train_loss, train_acc, val_acc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "86ed842a-bc9e-4371-a38e-555c28c60552",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.eval()\n",
    "test_path = glob.glob('./test/*.npy')\n",
    "test_loader = torch.utils.data.DataLoader(\n",
    "    XunFeiDataset(test_path, [0] * len(test_path),\n",
    "            A.Compose([\n",
    "            # A.HorizontalFlip(p=0.5),\n",
    "            # A.RandomContrast(p=0.5),\n",
    "            A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225))\n",
    "        ])\n",
    "    ), batch_size=30, shuffle=False, num_workers=1, pin_memory=False\n",
    ")\n",
    "pred = predict(test_loader, model, criterion)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "da338c21-ad72-43c2-9eb2-91f420e1e495",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.DataFrame({\n",
    "    'uuid': [x.split('/')[-1][:-4] for x in test_path],\n",
    "    'label': [labels[x] for x in pred.argmax(1)]\n",
    "}).to_csv('submit.csv', index=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a0fe32d6-5222-4403-986d-3756d08ee4a8",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py3.11",
   "language": "python",
   "name": "py3.11"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
